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ABSTRACT 


Cluster  analysis  involves  the  problem  of  optimal  partitioning 
of  a  given  set  of  entities  into  a  pre-assigned  number  of  mutually 
exclusive  and  exhaustive  clusters.  Here  the  problem  is  formulated 
in  two  different  ways  with  the  distance  function  (a)  of  minimizing  the 
within  groups  sums  of  squares  and  (b)  minimizing  the  maximum  distance 
within  groups.  These  lead  to  different  kinds  of  linear  and  non-linear 
(0-1)  integer  programming  problems.  Computational  difficulties  are 


discussed. 


CLUSTER  ANALYSIS  AND  MATHEMATICAL 
PROGRAMMING 

. .  .  M.R.RAO 

« 

INTRODUCTION 

Cluster  analysis  involves  the  problem  of  optimal  partitioning 

of  a  given  set  of  entities  into  a  number  (pre-assigned)  of  mutually 

1 

exclusive  and  exhaustive  clusters..  The  criterion  for  optimality 
depends  heavily  upon  the  application  in  which,  it  is  to  be  used. 

It  is  not  the  purpose  of  this  paper  to  discuss  the  relative  merits 
of  the  various  criteria.  A  discussion  of  this  can  be  found  in 
Sokal  and  Sneath  £l]. 

In  our  analysis  here*  we  confine  ourselves  to  distance  based 
cluster  analysis  in  which  a  distance  measure  between  the  various 
entities  is  available.  This  type  of  cluster  analysis  has  received 
increasing  attention  recently.  A  detailed  discussion  of  this  is 
given  by  Majone  [2]  and  Majone  and  Sandy  [3]  .  Even  though  a 
distance  measure  between  the  various  entities  is  known,  the  cri¬ 
terion  for  optimal  partitioning  still  depends  upon  the  intended 
application.  Again,  it  is  beyond  the  scope  of  this  paper  to  dis¬ 
cuss  the  relative  merits  of  the  various  possible  criteria  but  re¬ 
ferences  can  be  found  in  Jensen  [5]  where  a  dynamic  programming 
algorithm  is  given  for  minimizing  the  within  -  groups  sums  of 
squares.  . 

One  of  the  purposes  of  this  paper  is  to  consider  the  crite¬ 
rion  of  minimizing  the  within  groups  sums  of  squares  and  show  how 
the  problem  can  be  formulated  as  a  mathematical  programming  pro¬ 
blem.  In  the  general  case,  the  formulation  leads  to  a  fractional 
non-linear  , 


See  also  [Ujfor  an  application  of  distance  based  cluster  analysis 
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2. 


0-1  programming  problem  with  constraints  and  there  does  not  appear 
to  be  any  computationally  efficient  procedure  for  solving  such  a 
problem.  Fortunately,  the  problem  does  appear  to  be  computational¬ 
ly  tractable  in  some  variants  and  special  cases  which  are  discus¬ 
sed  in  detail. 

An  alternate  criterion , viz . ,  minimize  the  maximum  distance 
within  groups,  is  also  considered.  The  formulation  in  this  case 
leads  to  a  linear  integer  (0-1 )  programming  problem  which  can  be 
solved  by  any  one  of  the  known  techniques  [6,7]-  A  simple  but 
efficient  algorithm  is  given  to  solve  this  problem  when  the  number 
of  pre-assigned  groups  is  restricted  to  be  only  two. 

FORMULATION  OF  THE  PROBLEM. 

The  following  definitions  are  used  in  the  formulation  :  . - 

d- .  is  a  metric  distance  between  entities  i  and  j. 

H  is  the  number  of  entities 
M  i£  the  number  of  pre-ass  igne'd  groups. 

We  give  a  formulation  of  the  problem  under  different  criteria. 

1)  Minimize  (maximize)  the  within  (between)-groups  sums  of  squares 
Let  be  equal  to  1  or  0  depending  upon  whether  the  i^*1  entity 

..  is  in  group  k  or  not. 

Let  d.  .  be  a  Euclidean  metric'. 

*  J 

m-  The  problem  can  now  be  written  as 


M  N-1  N 

'Min  l  [(  Z  I 

k=1  i= 1  jsi+l 


d.  .  x..  x.,)/  I  x:_.  ] 
ij  ik  jk  .  .  lk 


M 

Subject  to  I  x..  =  1  for  i  =  1,2,...N  (2) 

k=1  1K 

L  0  and  integer  valued  for  i  =  1,2,...W 

k«  1,8,...M 
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This  is  a  fractional  non-linear  0-1  programming  problem  which 
“is  very  difficult  to  solve  in  general.  However  two  special  pases 
•of  this  appear  to  be  less  difficult  and  they  are  discussed  next. 


a)  The  number  of  entities  in  each  group  is  specified  : 

.  Sometimes  the  number  of  entities  in  each  group  is  specified 

in  advance.  Let  n.  be  the  number  of  entities  in  group  k  such  that 
M  k  -  - 

E  n  =  N . 
k=l  * 

The  objective  function  (1)  now  becomes 


M  N-l  N  2 

Min  E  - —  (  E  E  d.  . 

k=l  nk  i=l  j=i+l  0 


x..  x ) 
lk  jk' 


(3) 


and  -we  have  an  additional  set  of  constraints,  E  x. .  *=  n. 

i=l  * 


k  =  1 ,2,. . .M  ( U ) 


This  is  still  a  non-linear  0-1  programming  problem  with  con¬ 
straint  set."  (2)  and  (4).  However  since  all  n^  are  specified  in  ad¬ 
vance,  we  do  not  have  a  fractional  objective  as  in  ( 1 )  and  there 

s  •  .  **  *  , 

are  at  least  two  possible  approaches  to  attempt  to  solve  this  proble 
The  first  approach  is  to  treat  this  as  a  constrained  non-linear 
boolean  programming  problem  and  use  the  methods  outlined  by  Hammer 
and  Rudeanu  [8].  Another  approach  is  to  linearize  the  objective 
function  at  the  expense  of  increasing  the  number  of  constraints 
and  solve  the  resulting  problem  by  any  of  the  known  techniques  for 
linear  integer  programming.  The  0-1  programming  problem  to  be 
solved  is  as  follows  : 


M  N-1  N 

Min  E  —  (  E  E  d .  . 
k= 1  nk  i= 1  j=i+1  1J 


(5) 


Subject  to  xik  +  xJk. 


i  =  1 ,2  ... .N-1 
j  =  i+ 1 , i+2  , . . .N 


k  =  1  ,2  , .  .  .M 


(6) 


4.‘ 

H 

xik  c  nk  k  = 

i=1 

M 

£  x.  =1  i  =  1,2, ...N 

k=1  1K 

« 

All  and  y^j  _>  0  and  integer  valued. 

\ 

In  this  formulation,  the  number  of  constraints  increases  rapidly 
with  N  and  M  and  hence  this  approach  is  computationally  useful  only 
for  small  values  of  N  and  M. 

b)  The  number  of  pre-assigned  groups  is  equal  to  two  : 

A  method  for  cluster  analysis  suggested  by  Edwards  and  Cavalli- 
Sforza  [9]  involves  dividing  the  entities  into  two  most-compact 
clusters  and  repeating  the  process  sequentially.  This,  in  our 
notation^ implies  that  at  each  stage  of  the  process  we  have  M  =  2. 

In  this  case,  a  better  formulation  of  the  problem  is  possible 
witli  the  following  notation  : 

j.  t 

Let  itj  =  1  or  0  depending  upon  whether  the  1  entity  is  in  group 

1  or  2. 

The  problem  can  be  written  as 

•  ' 

H-1  $  p  N  K-1  N  - 

Mini (  l  l  df.  x.  x.)/  E  x.]  +  [{  Z  Z  df  .  ( 1 -x . ) ( 1 -x . ) } 

i=1  j  =  i+ 1  J  0  i  =  1  1  i= 1  j=i+ 1  1J  1  J 

N 

/(N  -  E  x.)]  (7) 

i=1  1 

x^  =  0  or  1  for  all  i. 

This  is  a  fractional  non-linear  0-1  programming  problem  with  no 
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additional  constraints . This  is  also  a  boolean  programming  problem  an 
one  approach  to  solve  this  problem  is  to  use  the  methods  given  in 
t  fi).  We  outline  two  other  methods  and  of  these  the  second  one 
.appears  to  be  promising.  It  should  be  pointed  out  however,  that 
so  far  n.o  computational  experience  is  available. 


1 )  In  the  first  approach,  we  rewrite  the  objective  function  .(7.) 
as  follows  : 

N  N-1  N  N  N-1  N: 

Min  [(N  -  £  x  .  )  (  £  £  dT .  x.  x.)  +  {  £  x.}{  £  £ 

i=l  i= 1  j  =  i+ 1  1J  1  J  i=l  1  i=1  j=i+1 


df.  (1-x. )(l-x. )}]/(  £  x. )(N  -  £  x.)  (8 


i=1 


i-1  1 


Let  the  objective  function  (8)  without  the  denominator  be 
referred  to  as  (8').  We  note  that  the  denominator  in  (8)  is  a 
maximum  if 


N 


rN- 


£  x .  =  [— ]  where 

i=1  1  2 

[^]  is  the  least  integer  greater  than  or  equal  to  N/2. 

We  first  minimize  the  objective  function  (8')  by  using  the 
methods  given  in  [8].  If  we  are  fortunate  to  obtain 

I  x  «=  [|]  or  [|]  -  1 
"•  i  =  1  1  2  2 

H  H  * 

the  problem  is  then  solved.  Otherwise,  let  £  x.  *=  m  <  [— ] . 

i=1  1  a 

Now,  we  know  that  the  only  possible  solutions  that  may  improve 

the  value  of  (8)  should  necessarily  satisfy  the  following  relation  : 


N 

•m  <  £  x .  <  N  -  -Jim 

i=1  1 


(9) 


There  is  no  loss  in  general ity  , in  assuming  that  m  < 

this  were  not  true  we  could  use  N  -  £  x.  instead 

i=  1  1 


,N,  . 

[m-J  since 
«  N 

of  £  x . 
i  =  1  1 


if 


6. 


The  next  step  is  to  consider  (8')  and  solve  the  problem  with  the 

added  constraint  (9)-  If  the  current  solution  has  a  "better  value 

for  the  objective  function  (8)  than  the  best  value  found  till  . 

now,  it' is  retained  as  the  best  value  found  thus  far.  The  process 

N  jj  '  t 

is  repeated  until  either  l  x.  *  [r]  or  the  value  of  the  function 

i=l  1  * 

(8')  becomes  so  high  that  the  value  for  the  objective  function  (8) 
would  be  higher  than  the  current  best  value  even  if  we  assume 


H 


I  x.  to  be  equal  to  [-]  0r  [jh  -  1 
i= 1  1  *  2 


The  efficiency  of  this  approach  depends  heavily  upon  the  abi¬ 
lity  to  find  the  solution  of  (8')  with  the  restriction  (9).  At 
present,  there  does  not  appear  to  be  efficient  methods  for  doing 
this. 


2)  An  alternate  approach  which"  appears  to  be  promising  is  to  use 
branch  and  bound  methods  to  solve  this  problem.  First  we  note 
that  the  problem  is  very  similar  to  the  problem  of  "selecting 
an  optimum  subset”  described  by  Beale  [10]  where  a  tree-search 
(branch  and  bound)  procedure  is  outlined.  Using  the  terminology 
in  [10],  we  briefly  describe  the  procedure  first  and  then  indicate 
in  some  detail  how  the  ^various  steps  can  be  performed  for  this 
problem. 


The  root  of  the  tree  corresponds  to  a  partial  solution  in 
which  all  the  elements  are  free  to  be  assigned.  We  select  one 
element  and  we  now  have  two  possibilities  of  assignment.  We  can 
assign  the  element  to  group  J  or  group  2.  These  two  possibilities 
correspond  to  the  two  branches  emanating  from  the  root.  For  con¬ 
venience,  we  will  always  write  the  assignment  of  an  element  to 
group  1  as  the  branch  to  the  right  and  always  branch  to  the  right 


For  convience  we  use  the  word  element  to  refer  to  an  entity. 
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.first.  Thus,  at  any  node  of  the  tree,  we  have  a  partial  solution 
with  some  elements  assigned  to  group  1 ,  some  assigned  to  group  2 
while  others  are  yet  to  be  assigned.  A  complete  solution  is  one 
in  which  all  the  elements  have  been  assigned  and  this  corresponds 
to  an  end  of  the  tree. 

Before  we  branch  from  a  particular  node,  we  compute  a  lower 
bound  for  the  objective  function.  If  the  lower  bound  is  greater 
than  or  equal  to  the  best  complete  solution  found  so  far,  we  do 
not  branch  further  from  this  node.  In  this  case,' we  back-track 
along  the  tree  until,  we  reach  a  node  with  an  unexplored  branch  to 
the  left.  If  no  such  node  exists,  the  problem  is  solved  and  the  be 
complete  solution  found  so  far  is  the  optimal  solution.  In  the 
other  case,  where  the  lower  bound  is  less  than  the  value  of  the 
current  best  solution,  we  need  to  branch  further.  We  select  a 
variable  not  yet  assigned  and  branch  to  the  right.  We  continue 
branching  until  we  reach  an  end  of  the  tree  or  the  lower  bound 
test  indicates  that  we  need  not  branch  further.  When  an  end  of 
the  tree  is  reached,  it  would  represent  an  improvement  in  the 
objective  function  and  hence  it  is  retained  as  the  current  best 
solution.  From  the  end  of  the  tree,  we  then  back-track  along  the 
tree  until  we  reach  a  node  with  an  unexplored  branch  to  the  left. 

SELECTION  OF  AN  ELEMENT  AT  A  NODE. 

♦ 

At  any  particular . node ,  we  have  a  set  (possibly  empty)  of 
elements  in  group  1  and  a  set  (possibly  empty)  of  elements  in  group 
2.  If  we  need  to  branch  further,  we  select  an  element  not  yet 
assigned  and  branch  to  the  right.  In  our  procedure,  this  corres¬ 
ponds  to  assigning  this  selected  element  to  group  1.  So  the  se¬ 
lection  of  an  element  is  made  such  that  the  sum  of  the  distances  be 
tween  this  element  and  all  other  elements  already  in  group  1  is 
minimum. 
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COMPUTATION  OF  LOWER  BOUND  AT  A  NODE. 

We  first  give  some  notations  for  convenience.  At  a  given  node 
let  .  * 

x^  =  1  -for  all  i  assigned  to  group  1 

*  0  for  all  i  assigned  to  group  2 

Let  Nj  =  {i/x^^  =  1},  and  Ng  *  {i/x^^  =  0}. 

Let  n^  and  ng  be  the  number  of  elements  in  N^  and  Ng  respectively. 

Let  n^  =  N  -  n^  -  ng  he  the  number  of  elements  not  yet  assigned 

to  any  group  and  let  N^  be  a  set  consisting  of  these  n^  elements. 

Let  K,  =  I  d. .  and  K  =  l  d. . 

1  *•>  2  12 

Let  D  be  a  N  x  N  symmetric  matrix  giving  the  distance  between  the 
entities.  Let  p  be  a'n^  x  n^  sub-matrix  of  D  giving  the  distance 
between  an  element  in  N^  and  an  element  in  N  ^ .  Let  be  the  sum 
of  the  elements  in  row  i  of  matrix  p. 

Let  .K  be  a  ng  x  n^  submatrix  of  D  giving  the  distance  between  an 

element  in  Ng  and  an  element  in  Nj.  Let  E^  be  the  sum  of  the 

elements  in  row  i  of  the  matrix  E- 

We  assume  that  matrices  E.&nd  p  are  arranged  in  such  a  manner 

that 


K  •  <  H  • 

l  —  j 

for  i 

<  j* 

f  .  <  V  . 

~  *  J 

for  i 

<  0  • 

Let  C  be  a  n'^  x  n2  symmetric  sub-matrix  of  D  giving  the  dis- 
.tance  between  each  pair  of  elements  in  N^.  The  diagonal  elements 
of  C  are  assigned  a  very  high  value  (<*>). 

We  are  interested  in  finding 


9. 


Z  =  Min  { 


K.  +  I  d . .  x .  x  .  +  £  d . .  x.  x . 

'■  i6Nj,j£.N3  0  1  J  3  3 

.  n,  +  £  x. 

. ‘**3 


Kg  +  £  d..(l -x . ) ( 1 -x . )  +  .£  d. .(1-x. )(l-x. ) 

2  itNg.’jfcl^  1J  1  J  i ,  j£N_  1J  1  J 

+  - - ^ - “ - — - — - ^ - 

-  n_  +‘ (n-  -  £  x. ) 

3  ifeN3 


(10) 


where  x^  =  0  or  1 


For  any  fixed  value  for  £  x.  *  ni  ,  let  n*  =  n_  “  nl , 
.  B  i£N3  1  3  3  3  3 

p  =  n^  +  n^  and  t  =  ng  +  n” .  Mow  we  can  write  (10)  as 


Z'  *=  Min  t[  £ 


Z  di .  x.  X.  +  £  d. .  x.  x.] 

1J  1  J  i,jtM3  10  1  J 


+  P  (  I  d.  .  (1-x.) (1-x.)  +  £  d. .(1-x. )(l-> 

•5  £.11  •?  d»T  ^  J  •  •  /  it  1J  1 

3 


i€N2,jfeN3  10  I'  '  y  i ,  j  tN.  ~i3 


Since  K^t  and  Kgp  are  constants  and  the  denominator  is  also 
a  constant  equal  to  pt . 


How,  there  are  several  ways  of  finding  a  lower  bound  for  Z' 
and  some  of  these  are  listed  below. 

By  varying  n^  over  its  range  0  to  n3 ,  we  obtain  n3  +  1  lower  bounds 
for  Z'  and  the  least  of  the  lower  bounds  gives  the  desired  bound 
for  Z  in  (10).  We  should  point  out  that  although  it  might  at  first 
appear  to  be  time  consuming  to  get  n3  +  1  lower  bounds  for  Z',  many 
of  these  calculations  are  very  easy  to  perform- Of  course,  without 
computational  experience,  it  is  difficult  to  say  how  effective  would 
be  the  bound  for  Z  obtained  in  this  manner. 
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i)  Z'  t  u,  +  u2  +  u3 

°3 

where  u.  =  (t)  Min  Z  d. .  x.  x.  =  (t)  Z  P. 

1J  1  J  i-1.1  - 

13  n3  . 

u.  =  (p)  Min  Z  d.  .(l-x. )(l-x.)  =  (p)  Z  H. 

i£N2,jeN3  1  J  i-1  .  1 

» 

u,  *  [(t)Min  Z  d‘.  .  x.  x.]  +  [(p)  Min  Z  d.  .  ( 1-x .  )  ( 1  -x  . )  ] 


i.jfeN, 


i,j£H3 


In  order  to  find  u3  we  first  note  that  the  summation  ( Z )  terms 
in  u3  consist  of  v  =  [n^(n^-1 )+n^(n£-1 ) ]/2  terms  in  either  the  upper3 
or  the  lower  triangular  part  of  the  symmetric  matrix  C.  Therfore 
a  lower  bound  for  u3  is  given  by  Min  [  t,  p]  multiplied 
by  the  sum-  of  the  v  smallest  elements  of  the  upper  half  of  matrix  C. 

ii)  Z'  >  +  u2  + 

where  u1  and  Ug  are  as  given  in  i)  and 

uu  =  Min  [t(  Z  d..  x.  x.)  +  p{.  Z  d.  .(1-x. )(l-x.)}] 

i,j«N_  1J  1  J  i,j€.N  1J  1  J 


Let  t  <_  p. 


u4  =  Min  [t(  l  d..{x.  xi  +  (1-x- )  ( 1-x  . )'}  )  +  (p-t)  Z  d.  .  ( 1-x.  )  ( 1-x  . 

i,jew3  ^  1  «>  1  J  i>j€N3  10  i  0 

’  >u5+u6 

where  u  =  t  Min  [  Z  d  .  {x.  x.  +  ( 1 -x . ) ( 1 -x . ) } ] 

i,0£N3  iJ  1  0  i  0 


We  will  take  the  upper  triangular  part  of  the  matrix  C. 

It 

If  t  >  p,  a. similar  procedure  would  hold  but  we  do  not  repeat' 
the  details  here. 


11. 


vbich  is  equal  to  t  multiplied  by  the  sum 
trix  C  (as  described  in  i.) 

=  (p-t)  Min  I  d. . ( 1 -x . ) ( 1 -x . ) 

6  i,jCW3  13  1  3 

In  order  to  find  Ug  ve  note  that  the 
of  r  =  n3(n3-l)/2  terms  in  the  upper  half 
bound  for  Ug  is  given  by  (p-t)  multiplied 
elements  of  the  upper  half  of  C. 

iii)  Z'  2:  Uj  *  (u^  or  u^) 

where  u^  and  u^  are  as  given  in  i)  and  ii)  respectively  and 

u„  =  Min  [t(  I  d.  .  x.  x.)  +  p{  I  d.  .  ( 1 -x . ) ( 1 -X  . ) } ] 

'  itN1tj£N3  13  1  3  i£N2,jCN3  J  J 

In  i)  we  used  a  lower  bound  of  Uj  +  Ug  for  u^.  A  better 

lower  bound  can  be  obtained  for.  u^  by  solving  a  transportation 

problem  [IV]  as  indicated  below.  Let  the  elements  of  if,  be  num- 

i  ^ 

bered  as  1,2,...  n3»  Let  J  be  the  sum  of  all  the  elements  of  a 
row  in  F  corresponding  to  element  i  in  Nj.  Similarly,  let  Hl 
be  the  sum  of  all  the  elements  of  a  row  in  H  corresponding  to  ele¬ 
ment  i  in  Hj. 

How,  we  have  to  subdivide  the  n3  elements  of  N3  into  two  sub' 
sets  one  consisting  of  n^  elements  and  the  other  consisting  of 
i»3  elements.  The  two  divisions  can  be  considered  as  two  demand 
points,  the  first  one  requiring  n'  units  and  the  other  requiring 

J  > 

n3  units.  The  n3  elements  can  be  considered  as  n3  supply  points 
each  with  a  supply  of  one  unit.  The  cost  of  transporting  a  unit 


of  the  v  terms  of  the  mi 


summation  term  consists 
matrix.  C.  So  a  lower 
by  the  r  smallest 


12. 


from  the  i^*1  supply  point  to  the  first  and  second  demand  points 
is  given  by  t  B1 ' and  jj  A1  respectively.  The  minimum  transporta¬ 
tion  cost  gives  Uj  which  is  a  better  bound  than  u^  +  Ug.  Of  cours 
in  order  to  obtain  this  better  bound,  we  have  to  solve  a  transpor¬ 
tation  problem  which  can  be  time  consuming  since  it  has  to  be  re¬ 
peated  n^  +  1  times  to  obtain  a  bound  for  Z.  .  iBu't  *  some  of 
these  (for  e.g.  n^  =  0  or  n^  =  n^  etc.)  are  trivial  calculations 
and  do  not  require  a  transportation  problem  to  be  solved. 


The  efficiency  of  the  branch  and  bound  procedure  outlined  ' 
here  depends  upon  the  effectiveness  of  the  lower  bound  at  a  node 
which  is  difficult  to  predict  without  computational  experience. 
An  additional  feature  described  in  [10]  is  the  restructuring  of 
the  tree  and  this  can  be  incorporated  here  also.  However,  here 
we  omit  the  details  which  are  given  in  [10], 


II.  Minimize  the  total  within  group  distance. 


If  we  are  interested  in  only  minimizing  the  total* within 

groups  distance^,  the  objective  function  (3)  would  be  modified 

2 

so  that  the  constant  term  1/n,  does  not  appear  and  the  term  d. . 

h -  **  ij 

is  replaced  by  d...  As  before,  the  constraint  set  is  given  by 

1  J  •  _ 

(2).  If  necessary,  by  adding  the  constraint  (6)  we  can 


If  the  total  between  groups  distance  is  defined  to  be  the  sum 
of  all  the  distances  between  entities  that  do  not  belong  to  the 
same  group,  then  minimizing  the  total  within  groups  distance  is 
equivalent  to  maximizing  the  total  between  groups  distance. 
Cooper  and  Majone  [12]  suggest  a  criterion  of  minimizing • (naxini 
zing)  the  total  within  (between)  groups  distance  subject  to  a 
constant  level  of  between  (within)  cluster  distances.  It  should 
be  possible  to  obtain  a  constant  level  of  between  cluster  dis¬ 
tances  by  adding  appropriate  constraints. 


linearize  this  new  objective  and  obtain  a  function  very  similar 
to  that  given  in  ($). 

If  the  number  of  groups  (M)  is  restricted  to  be  two,  the 
objective  (7)  would  now  become 

N-1  K  N-1  N 

Min  (  I  Z  d.  .  x.  x.)  +  {  Z  Z  d . .  ( 1-x. ) ( 1-x. ) } 

i= 1  j=i+ 1  J  J  i= 1  j=i+1  1J  1  J 

Since  the  only  constraints  in  the  problem  are 

.  x^  *=  0  or  1  for  all  i 

this  problem  can  be  solved  without  difficulty  by  the  methods 
given  in  [8] •  ; 


II T.  Minimize  the  maximum  within  group  distance. 


With  this  criterion,  instead  of  minimizing  the  total  within 
group  distance,  we  would  minimize  the  maximum  distance  within 
groups.  A  mathematical  formulation  of  the  problem  is 

Min  Z 


Subject  to  d.  .  x.,  +  d.  . 
•  ij  lk  ij 


Z  <  d.  . 
-  ij 


M 

Z 

k=  1 


i  =  1  ,2 


i  *=  1,2,..  .N-1 
j  =  i+1  ,i+2 .  .  . N 
k  =  1  ,2,.  .  .M 


.N 


1 


and  Z  >.  0  and  integer  valued 


This  is  an  integer  linear  programming  problem  but  unfor¬ 
tunately  the  number  of  constraints  increases  very  rapidly  with 
H  and  M  and  hence  this  formulation  is  computationally  useful  only 
for  small  values  of  N  and  M. 

•  .  x 

It  should  be  pointed  out  that  if  ve  solve  this  problem  as  a 
linear  programming  problem  without  the  variables  being  restricted 
to  integers,  the  solution  is  x^  -  ^  for  all  i  and  k.  Hence  we 
vould  never  be  fortunate  to  obtain  an  integer  solution  as  the 
linear  programming  solution.  However  when  the  number  of  groups 
(M)  is  equal  to  two,  this  problem'  can  be  solved  efficiently  as 
outlined  below. 

• 

.a)  Number  of  groups  (M)  equal  to  two  : 

Let  D  be  the  matrix  giving  the  distance  d..  between  the 

*  j 

entities  i  and  j. 

Let’  A  and  B  refer  to  the  two  groups.  If  an  element  receives  a 
label  A '(b)  it  means  that  it  is_  assigned,  permanently  to  group  A  (B). 
A  label  k  (integer  between  1  andH)  to  an  element  is  a  temporary 
label. 

'  • 

An  algorithm  to  solve  the  problem  is  given  next  and  a  flow 
chart  is  provided  in  figure  1. 


Without  any  loss  in  generality  wemay  assume  that  all  d..  are 
positive  integer  valued  and  hence  Z  will  also  be  an  integer. 


15. 


7 

1.  Let  d. -  be  the  largest  value  in  the  matrix  M. 

2 ;  i  receives  a  label  A  and  j  receives  a  label  B. 


3.  Let-d. , 
*  J 


8 

-1.  y  Let  d  be  the  current  largest  value  in  M. 

•  P4 

Let  i  =  P'  and  j  =  q.  If  both  i  and  j  have  not  received  any  la¬ 
bel  go  to  4.  Otherwise  go  to  5*  ' 

4.  Assign  i  a  label  i  and  j  a  label  i.  Go  to  3. 

5-  If  either  i  or  j  but  not  both  has  received  a  label  A  or  B  go 
to  6.  Otherwise  go  to  8.  ‘ 

6.  If  j  has  a  label  A  or  B  go  to  7*  Otherwise  i  has  a  label  A  (B) 
but  j  does  not  have  a  label  A  or  B.  If  j  does  not  have  a 
label,  label  j  as  B  (A)  and  go  to  3.  If  j  has  a  label  k,  give 
a  label  B  (A)  to  all  elements  with  label  k  and  a  label  A  (B) 
to  all  elements  with  label  k.  If  j  has  a  label  k,  give  a 
label  B  (A)  to  all  elements  with  label  k  and  a  label  A  ( B )  to 
all  elements  with  label  k.  Erase  the  labels  k  and  k.  Go  to  3. 
7-  j.has  a  label  A  (B)  but  i  does  not  have  a  lable  A  or  B.  If  i 
does  not  have  a  label,  label  i  as  B  (A)  and  go  to. 3.  If  i  has 
a  label  k,  give  a  label  B  (A)  to  all  elements  with  label  k  and 
a  label  A  (B)  to  all  elements  with  label  k.  If  i  had  a  label 
k,  give  a  label  B  (A)  to  all  elements  with  label  k  and  a  label 
A  (B)  to  all  elements  with  label  k.  Erase  the  labels  k  and  k. 


Go  to  3. 

8.  If  both  i  and  j  have  the  same  label  A  (B)  go  to  16.  If  i  has 
a  label  A  (B)  and  j  has  a  label  B  (A)  go  to  3-  Otherwise  go 
to  9  ♦ 


7 

In  case  of  ties,  for  unambiguity  choose  the  one  with  the  smalles 
index  i.  To  break  ties  further,  if  any,  choose  the  smallest 
index  j .  • 

8 

We  have  assumed. that  all  d  are  non-negative.  If  some  d 

P<1  P<1 

negative,  we  would  set  d .  .  equal  to  a  large  negative  number. 

*  J 


are 


16. 


9«  Both  i  and'j  do  not  have  a  label  A  or  B.  If  either  i  or  j 
hut  not  both  has  been  labeled  before,  go  to  10.  Otherwise 
go  to  12. 

10.  If  j  has  a  label  k(k)  go  to  11.  Otherwise  i  has  a  label.  Let 
the  label  of  i  be  k(k).  Give  j  a  label  k(k) .  Go  to  3. 

11.  Let  the  label  of  j  be  k(k).  Give  i  a  label  k(k).  Go  to  3. 

12.  Both  i  and  j  have  a  label  (but  neither  of  them  has  a  label 

A  or  B).  If  both  i  and  j  have  the  same  label  k(k)  go  to  16.  t 
Otherwise  go  to  13. 

13.  If  i  is  labeled  k(k)  and  j  is  labeled  k(k)  go  to  3.  Otherwise 
go  to  n. 

14.  i  has  a  label  s(s).  If  j  has  a  label  t  go  to  15-  Otherwise 
j  has  a  label  t.  Assign  a  label  s(s)  to  all  elements  with 
label  t  and  assign  a  label  s(s)  to  all  elements  with  label  t. 
Erase  the  labels  t  and  t.  Go  to  3. 

15*  "Assign  a  label  s(s)  to  all  elements  with  label  t  and  assign 
a  label  s(s)  to  all  elements  with  label  t.  Erase  the  labels 
t  and  t.  Go  to  3> 

16.  The  minimum  of  the  maximum  distance  within  groups  is. equal  to 

d . . .  For  k  =  1,2,...N,  all  elements  with  label  k  are  assigned 

*  J  ^ 

a  label  A  and  all  elements  with  label  k  are  assigned  a  label  B. 
Erase  all  labels  k  and  k.  Remaining  elements,  if  any,  that 
have  not  been  examined  so  far  can  be  assigned  a  label  A  or  B 
arbitrarily^.  EHD 


9 

We  have . assumed  here  that  we  are  interested  in  only  minimizing 
the  maximum  within  group  distance.  A  better  approach  would  be 
to  first  assign  some  of  the  elements  to  the  two  groupjj.so  that 
the  maximum  within  group  distance  is  minimized.  Once-'Ts  achieve 
other  elements  may  be  assigned  to  the  two  groups  so  that  the 
minimum  distance  (considering  only  those  elements  not  yet  assig¬ 
ned)  between  groups  is  maximized. 


Give  a  label  B  (A)  to 
all  elements  with  label  k 
and  a  label  A  (B)  to_all 
elements  with  label  k. 
Erase  labels  k  and  k. 


i  has  a  label  k 
give  a  label  B  (A) 
to  all_elements  with 
label  k  and  a  label 
A  (B)  to  all  elements 
with  label  k.  j 

Erase  labels  k  and  k.  ' 


Figure  1 

Flow  chart  for’the  algorithm. 
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The  minimum  of  the  max. 
distance  =  d . • • 

For  k  =  1  ,2 ,T . .N  ,  all 
elements  vith  label  k 
are  assigned  a  label  A 
and  all  elements  with 
label  k  are  assigned 
a  label  £. 

Erase  all-  labels  k  and 
k. Remaining  elements, if  any, 
are  assigned  arbitrarily 
to  A  or  B. 


Yes 


i  has  a  label 
A  ( B )  and  j  has 
a  label  B  (A) 


Yes 


Yes/T 


<  Either  i  or  j 
but  not  both 
has  a  label 


has  a  label 
k(k) 


> 


Mo 


Label  i  as 
k(k) . 


Yes 


< 


<?  ^ 
Both,  i  and  j  have 
the  same  label 


> 


i  has  a  label 
k(k) . _  Label 
j  as  k (k ) . 

1 

k 

Mo 


No 


Yes' 


— — - \ 

|i  has  a  label  s(s) 

has  a  label  t 


> 


i  has  a  label  k(k) 
and  j  has  a  label 
k(k) . 


No 


Yes 


© 


Assign  a  label  s(s)  to  all 
elements  with  label  t  and 
assign  a  label  s(s)  to  all 
elements  with  label  t:  Erase 
the  label  t  and  t. 


j  has  a  label  t.  Assign  a 
label  s(s)  to  all  elements  wi 
label  t  and  assign  a  label  s( 
to  all  elements  with  label  t. 
Erase  the  labels  t  and  t. 


© 


© 


Flow  chart  for  the  algorithm  (contd.) 


■!->»  Vi 
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Flow  chart  for  the  algorithm  (contd.) 


CONCLUSIONS . 

In  "this  paper  we  have  shown  how  some  of  the  problems  in 
distance-based  cluster  analysis  can  be  viewed  as  a  mathematical 
programming  problem.  Many  of  the  problems  are  v:ery  difficult  to 
solve  and  require  further  research  in  terms  of  solution  techniques 
But  as  we  have  indicated  here,  the  problem  of  minimizing  the  maxi¬ 
mum  within  group  distance  is  easily  solvable  when  the  number  of 
groups  is  restricted  to  two.  Some  of  the  other  problems  appear  to 
be  computationally  tractable  and  deserve  some  computations  expe¬ 
rience  before  drawing  further  conclusions. 
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